Human Activity Recognition in Videos

Our project deals with the problem of classifying real-world videos by human activity. Such videos usually have a large variation in background and camera motion. This makes the performance of models using low-level appearance and motion features unsatisfactory, particularly in the case of video classes sharing similar objects and background (e.g. ”snatch” and ”clean-jerk” weightlifting actions). Objects present in a video, and the event label are mutually related. For instance, the presence of a “barbell” in a video would help classify it as a “weightlifting” event. Similarly, we would expect to see a “barbell” in a “weightlifting” video . Thus, recognizing the object presence and motion in a video should aid the event classification task. However, this requires an accurate detection of object tracks, which is a highly challenging task in itself. Works like [12], use humans in the loop to obtain good quality object tracks. However this requires significant human effort. We address this issue by extracting candidate tracks from a video, and modeling the choice of correct tracks as latent variables in a Latent SVM (LSVM) [14]. This formulation enables us to peform action recognition and weakly supervised object tracking in a joint framework. This leads to a more robust as well as discriminative choice of object tracks for event classification. Candidate object tracks are extracted using Deformable Part based Models (DPM) [2] and a tracking algorithm from [12]. We capture the object appearance and motion in a video through features extracted from the object tracks to use in our LSVM model. Finally, we test the performance of our model against different baselines, and show imporvement over the stateof-the-art method on the Olympic Sports Dataset introduced in [7]. 2. Related work